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SUMMARY 


This  paper  considers  a  number  of  possible  ways  of  implementing  the  Ada 
rendezvous  in  a  computer  system  comprising  a  number  of  processors.  It  shows 
that,  in  principle,  a  two  phase  protocol  requiring  four  messages  to  be  passed 
is  necessary  to  implement  the  rendezvous  correctly  when  timed  calls  are  used. 
However  in  many  cases  a  simpler,  one  phase,  protocol  requiring  only  two 
messages  per  rendezvous  can  be  used. 

A  coiq>arison  is  made  between  the  rendezvous  and  message  based 
comnunication  for  situations  where  one  to  many,  rather  than  one  to  one, 
coimmmication  is  required.  The  rendezvous  is  shown  to  be  very  inefficient 
for  implementing  one  to  many  conmunication.  Finally  some  of  the  problems  of 
loading  and  running  Ada  programs  are  briefly  considered, 
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Tntroiuction 


The  purpose  of  this  iocunent  is  to  discuss  the  pcoblenis  of  implemaitirKi 
Ada  on  a  ^Ijltiple  Processor  Cbimxiter  f^tein  and  to  coirpare  possible  ways  of 
overcominq  these  problems.  We  shall  be  concerned  primarily  with  how  a 
rendezvous  can  be  achieved  bet%#een  tasks  runninq  on  different  computers,  as 
this  seems  to  be  the  main  technical  ^aroblem.  We  will  compare  the  different 
ways  of  imp^ementinq  the  rendezvous  in  terms  of  the  nunnber  of  inter  - 
computer  messaqes,  an^  context  switches  which  they  require  as  these  are 
k  /pr^ably)  the  two  most  important  aspects  of  the  overheads  involved  in 

providinq  the  rendezvous. 

Assumptions 


We  make  the  foUowinq  basic  assumotions  ^ut  the  characteristics  of  the 
Multiple  Processor  System  fMPS)  on  vAiich  we  vnsh  to  implement  Ada.  First,  it 
seems  that  the  implementation  of  the  rendem^ous  in  a  system  havinq  ^ared 
main  memory  can  be  2K:hieved  simply  by  extending  a  single  processor 
implementation.  Tt  is  less  obvious  how  to  implanent  the  rende»rous  without 
shared  main  memory,  so  we  will  concentrate  our  attention  on  such  Multi 
Computer  Systems  (HCSl .  As  a  consequence  we  will  assume  that  paraneters  and 
results  are  passed  by  value  rather  than  by  reference. 

Second,  we  assume  that  there  is  a  kernel  running  in  each  computer  which 
supports  the  Ada  tasks,  performs  scheduling  and  provides  the  mechanisms  for 
inter  -  computer  communication.  We  assume  that  the  kernel  need  not  be 
written  in  Ma. 

Third,  we  assume  that  the  communication  system  provides  the  abstraction 
of  total  reliability,  so  we  will  not  consider  the  problems  of  lost  messages 
etc.  Tjmolicitly  this  means  that  the  transmission  of  a  single  message,  at  the 
user  program  level ,  may  require  many  low  level  messages  to  be  transmitted . 
Mb  will  always  quote  the  number  of  messages  required  by  a  particular 
implementation  of  the  rendezvous  at  the  user  program  level  as  this  gives  tJie 
simplest  basis  for  comparison. 

Finally,  we  distinguish  two  different  types  of  context  switch.  A  Task 
Context  5?witch  (TCS)  occurs  *hen  a  task  is  scheduled  or  suspended  for  some 
reason  (e.g.  waiting  for  an  entry  call  to  be  accepted).  An  Interrupt  Ccwjtext 
Switch  (tcS)  occurs  vhen  an  interrupt  handler  is  entered  or  exited.  We  will 
assume  that  an  TCS  only  occurs  tvnce  per  received  message  although  this  may 
be  very  far  from  the  truth  with  word  or  byte  oriented  communication  systems. 
One  might  exoect  that  the  time  required  to  perform  an  TCS  will  be  smaller 
than  that  required  to  perform  a  TCS,  although  this  will  not  universally  he 
true. 


Tt  is  perhaps  worth  noting  that  the  Mass!  -  Habermann  optimisation 
cannot  be  performed  vhen  the  communicating  tasks  are  in  different  computers. 
However  it  might  be  possible  to  exploit  this  optimisation  if  shared  main 
memory  is  available. 

3)  Simple  Rendezvous 

Hy  a  sdmole  rendezvous  we  mean  one  ^re  the  calling  task  calls  the 
entry  unconditionally  (and  without  timeouts),  and  the  called  task 
•  unconditionally  accepts  tihe  call  (although  not  necessarily  innediately  it  is 

issued).  Haturally  we  assume  that  the  calling  and  called  tasks  are  running 
on  separate  computers.  The  simple  rendezvous  can  be  implemented  as  indicated 
below: 
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Calling  Task 


Called  Tcisk 


entryj^alT 

suspend 


_return 

suspend 


Ihis  straightforward  implementation  requires  two  messages  to  be  passed 
between  the  computers,  and  requires  two  IC?  in  each  machine  to  handle  the 
receipt  of  the  messages.  Tn  the  calling  task  two  ICS  are  required.  In  the 
called  task  up  to  t«io  ICS  may  be  required,  but  the  number  depends  on  whether 
or  not  the  task  is  active  ininediately  before  and  after  the  rende^ous.  This 
comment  applies  throughout  the  following  discussion. 

Tf  necessary  exceptions  can  be  returned  bo  the  calling  task:  e.g. 
TSsking^^Error  can  be  returned  if  the  called  task  has  terminated. 

41  Conditional  Entry  Calls 

These  can  be  implemented  in  the  same  basic  way  as  the  simple  rendezvous, 
except  that  a  rejection  may  have  to  be  returned,  rather  than  the  result  of 
executing  the  entry  procedure.  Tt  is  possible  tdiat  two  TCSs  will  be  required 
in  the  called  task  even  if  the  call  is  rejected.  These  context  switches  can 
be  avoided  if  the  kernel  (interrupt  handling  routines)  can  preserve  enough 
information  about  the  called  task  to  know  %*iether  or  not  a  rendezvous  is 
oossible,  without  entering  the  environment  of  the  called  task. 

51  Selective  Waits 


schedule 

return 


return  msg- 


^call  msg. 


.schedule 

accept 


Selective  waits  can  be  implemented  using  the  basic  method  described 
above. 

5)  Timed  Entry  Calls 


5.1)  Introduction 


If  timed  ^try  calls  were  implemented  by  the  above  mechanism  it  is 
possible  that  the  timeout  might  expire,  causing  the  calling  task  to  execute 
its  alternative  code,  vbilst  the  call^  task  was  executing  tiie  entry.  This 
eventuality  would  be  a  violation  of  the  rendezvous  semantics,  and  clearly 
must  be  avoided. 

Fbr  a  timed  entry  call,  the  timing  is  performed  on  acceptance  of  the 
entry  call,  not  on  the  execution  of  the  entry.  This  means  that  (in  principle 
at  least)  information  needs  to  be  passed  back  to  the  calling  task  once  the 
call  is  accepted,  as  well  as  on  the  completion  of  the  call.  This  is  the 
basis  of  our  first  alternative  solution  below. 

5.2)  Simple  Approach 


.1 


The  simplest  approach  to  implementinq  timei  entry  calls  seems  to  be  to 
use  a  two  chase  "hani^ake*  protocol .  The  first  phase  governs  the  acceptance 
of  the  call,  ani  the  seconri  is  equival«it  to  the  protocol  'tescribei  in 
section  1  controlling  the  execution  of  the  entry.  The  acecution  of  the 
protocol  would  be  as  follows  if  the  call  vete  successful: 


Callinq  Thsk 


Called  Task 


entry__call . 
suspend 


schedule, 
calljconfirm 
suspend 


schedule,, 

return 


•call  msg. 


return  msg^ 


schedule 
accept 

report_accept 

suspend 


schedule 
execute  entry 
return" 
suspend 


This  requires  four  messages,  four  TC?  for  each  task,  and  four  ICS  for  each 
task. 


Because  of  the  timeout  it  is  possible  for  the  called  task  to  accept  the 
entry  call  after  the  callers  timeout  has  expired,  and  the  task  has  continued 
execution.  There  are  two  ways  of  recovering  from  this  situation:  the  calling 
task  can  be  "roUed  back",  so  that  the  entry  is  executed?  or  the  acceptance 
of  the  call  can  be  cancelled.  The  former  course  may  be  very  difficult, 
especially  if  the  calling  task  has  entered  into  communication  with  other 
tasks,  or  performed  some  I/D  before  the  accept  message  is  received.  The 
latter  course  oi.ly  requires  the  information  recording  the  acceptance  of  the 
call  to  be  changed  (eis  the  called  task  will  be  suspended) .  Clearly  the 
latter  approach  is  much  simpler  to  achieve  and  it  correctly  implements  the 
semantics  of  the  timed  call. 

The  behaviour  will  be  as  follows  for  a  call  which  is  not  accepted  before 
the  timeout  expires: 

Calling  Task  Called  Task 

entry-call 
suspend 

I 
I 
1 
I 

time  out 
schedule 
abort  entry 

T 


This  requires  three  messages,  two  TCS  in  the  calling  task,  and  fPur  in  the 
called  task.  There  will  be  two  ICB  in  the  calling  task,  and  four  in  the 
called  task. 


call  msg 


schedule 
accept 

report  accept 
su^end 


schedule 
canoel^accept 
su^nd 
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A  problem  which  arises  with  this  implementation  is  dealing  with  accept 
messages  i^ich  arrive  long  after  the  timeout  has  extairei,  ani  the  calling 
task  has  continuei  with  its  alternative  action.  There  seem  to  be  two 
possible  approaches.  The  kernel  could  maintain  information  regarding  the 
incomplete  rendezvous  until  the  accept  message  is  received  and  the  abort 
message  can  be  gaierated.  This  technique  may  give  problems  with  storage 
management  if  large  numbers  of  rendezvous  have  to  be  "remembered". 
Alternatively,  the  incomplete  rendezvous  can  be  forgotten  and  the  kernel  can 
respond  to  any  unrecognised  accent  message  with  an  abort  message.  This 
latter  technique  will  not  impose  storage  cwerheads  but  it  has  the 
disadvantage  that  it  will  not  help  in  the  detection  of  certain  error 
conditions,  such  as  trying  to  communicate  vnth  a  task  that  has  already 
terminated . 

Clearly  this  straightforward  implementation  is  quite  costly  in  terms  of 
messages  and  context  ^fitches.  There  are,  however,  more  efficient  ways  of 
implementing  the  rendezvous  which  may  be  applicable  in  some  circumstances. 
These  methods  rely  on  being  able  to  execute  l±e  timeout  in  the  called  task. 

Timing  at  Both  Ends 

When  calling  the  entry,  the  calling  task  could  transmit  the  timeout 
duration  to  the  called  task.  Assuming  that  both  tasks  have  access  to  clocks 
running  at  roughly  similar  rates,  then  the  called  task  can  inspect  the 
timeout  period  and  not  reply  if  it  knov®  that  it  will  not  be  able  to  accept 
the  call  quickly  enough,  unfortunately  we  cannot  guarantee  to  avoid  the 
situation  vbere  an  accept  message  is  received  after  the  callers  timeout  has 
expired  (unless  the  timing  of  the  comnunication  etc.  is  deterministic  and 
well  known!  so  we  must  still  cater  for  the  possibility  of  having  to  abort 
the  rendezvous. 

6.4)  Timinq  only  at  Called  End 

Tf  the  message  delay  through  the  communication  system  is  well  known, 
deterministic,  and  diort  with  respect  to  the  timeout  period  then  it  may  be 
possible  for  the  called  task  to  execute  the  timeout  on  behalf  of  the  calling 
task.  If  this  is  the  case  then  we  can  use  a  single  phase  protocol  for 
oer forming  the  rendezvous  as  we  described  in  section  3.  The  only  change  to 
the  protocol  of  section  3  is  that  the  called  task  can  return  a  "timed  out" 
message  to  the  caller  in  order  to  allow  it  to  continue  without  performing 
the  rendezvous. 

This  implementation  is  attractive  in  that  it  is  comparatively  efficient. 
However  there  is  a  problem  to  do  with  reliability.  If  the  processor  running 
the  called  task  fails  then  the  calling  task  may  never  (or  v«ry  belatedly) 
receive  its  timeout  message.  Oie  of  the  reasons  for  using  timed  entry  calls 
may  be  to  allow  detection  of,  and  recovery  from,  remote  failures,  in  which 
case  timing  at  the  called  end  may  not  be  satisfactory.  Arguably  it  Should  be 
the  responsibility  of  the  kernel  to  detect  remote  failures  and  to  return  a 
suitable  exception  to  the  task.  However  tiie  time  taken  by  the  kernel  to 
detect  the  failure  may  be  much  longer  thm  the  time  the  calling  task  is 
willing  to  wait.  It  seems  therefore  that  there  will  be  circumstances  under 
which  the  two  phase  protocol  trill  have  to  be  used. 

"')  Nummary  of  Rendezvous  iwplereentations 

we  have  described  a  mater  of  ways  in  which  the  rendezvous  can  be 
implemented.  The  most  efficient  method  t^ich  we  can  use  is  a  simple,  one 
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Dhase  protocol  as  'lescribed  in  section  6.4.  Ibis  orotocol  can  cater  for  the 
simple  rendezvous r  conditional  rendezvous  and,  under  some  circumstances, 
with  timed  rendezvous.  This  orotocol  will  tvoically  require  two  messaqes, 
two  TCS  per  task,  and  two  ICS  per  task. 

FPr  the  circunstances  v4iere  the  one  phase  protocol  is  not  acceptable, 
e.q.  *bere  we  wish  to  recover  from  the  failure  of  remote  computers,  then  the 
two  phase  protocol  described  in  section  6.2  will  have  to  be  used.  This 
protocol  has  twice  the  overhead  of  the  single  phase  orotocol  if  the 
rendezvous  is  completed  successfully.  If  the  rendewous  is  not  completed 
(due  to  f.  timeout  expiring)  then  the  overheads  are  rather  lower. 

B)  Comparison  with  Message  Passing 

Message  Passing  Paradigms 

There  are  essentially  three  distinct  forms  of  message  passing  inter 
process  communication  scheme.  The  simplest  simply  consists  of  the  sender 
transmitting  a  message,  and  continuing  execution  without  an  acknowledgement 
ever  being  returned.  This  scheme  does  not  make  it  easy  to  detect  remote 
failures  or  lost  messages,  but  may  be  quite  api»ropriate  under  certain 
circumstances  -  e.g.  transmission  of  data  from  a  sensor,  where  the  loss  of 
the  occassional  reading  will  not  adversely  affect  the  behaviour  of  the 
system. 

The  second  paradigm  is  that  of  one  process  sending  a  message,  then 
waiting  until  the  message  is  acknowledged  by  the  process  vhich  received  the 
message.  This  method  gives  scope  for  detecting  aid  recovering  from  certain 
classes  of  errors  (e.g.  lost  messages  caused  by  communications  failures), 
and  clearly  matches  the  semantics  of  the  rendezvous. 

The  third  paradigm  is  that  of  sending  a  message  and  receiving  an 
explicit  acknowledgement,  with  the  sending  process  able  to  continue 
processing  between  sending  the  message  and  receiving  the  acknowledgement. 
The  advantage  this  offers  over  the  second  method  is  that  concurrency  is 
improved  since  the  sending  process  may  be  performing  useful  work  v4iilst 
waiting  for  an  acknowledgement.  This  form  of  behaviour  can  only  be  echieved 
in  Ada  by  the  artifice  of  creating  a  task  specifically  for  performing  the 
inter  -  task  communication,  thus  allowing  the  parent  task  to  continue 
executing.  This  technique  can  have  severe  disadvantages  as  described  below. 

8.2)  The  Two  Phase  Commit  Protocol 


vfe  describe  the  implementation  of  the  TVo  lhase  Oommit  Protocol  in  Ma 
as  it  serves  bo  show  the  problems  v4iich  arise  from  the  fact  that  a  task 
cannot  continue  executing  between  sending  a  message  and  receiving  an 
acknowledgement  (i.e.  the  task  is  suspended  ii^lst  the  rendezvous  is  in 
progress) .  vte  believe  this  protocol  to  be  a  very  salient  example  as  it  is 
widely  used  as  a  «fay  of  ensuring  oonsistency  control  in  distributed  database 
systems. 

Mb  can  adequately  describe  tiie  most  important  features  of  the  TWO  lhase 
Oommit  Protocol  (2PC)  by  the  following  examole.  Imagine  that  we  have  a 
replicated  database  \Ath  a  total  of  N  copies,  and  we  wish  to  upleite  the 
database  so  that  all  the  copies  remain  in  step.  In  essence  we  need  to  update 
all  the  copies  indivisibly.  This  is  achieved  by  one  task  (the  control  task) 
notifying  all  the  copies  that  an  i^ate  is  to  be  performed,  and  tiie  tasks 
responsible  for  the  copies  either  acknowledge  that  they  can  perform  the 
update,  or  say  that  the  update  has  to  be  aborted  because  it  conflicts  with 


some  update  already  in  progress.  This  is  the  first,  or  notification,  phase. 
T'je  originating  task  then  either  informs  all  the  cooperating  tasks  that  the 
update  must  be  aborted,  or  instructs  them  to  perform  the  update,  as 
aopro^iate.  Tn  the  latter  case  the  tasks  %»ill  update  their  copies  of  the 
dat^ase,  then  return  acknowledgements  to  the  originating  task.  This  is  the 
second,  or  update,  phase. 

Clearly  the  implementation  of  the  protocol  will  be  complicated  by  the 
need  to  deal  with  failures  of  remote  computers  etc.,  but  the  basic  form  is 
not  affected  by  these  considerations.  Tlsing  our  third  paradigm  for  the 
message  passing  model,  the  2PC  control  task  can  transmit  messages  to  all  N 
tasks,  then  wait  for  acknowledgements,  in  both  the  first  and  second  phases. 
Thus  the  message  passing  discipline  allows  us  to  achieve  a  high  degree  of 
concurrency. 

Tf  we  implemented  the  ?PC  protocol  in  Wa  the  simplest  way  would  be  to 
perform  the  communication  with  each  of  the  N  cooperating  tasks  in  turn,  thus 
having  K  rendezvous  in  series  in  the  first  phase,  and  similarly  for  the 
second  phase.  Nbte  that  this  already  requires  twice  as  many  messages  and 
context  switches  as  the  messape  based  implementation.  This  implementation 
will  Pbviously  be  slow  as  the  inherent  parallelism  is  lost. 

Tn  order  to  itmxove  rarallelism  we  could  implement  the  protocol  so  that 
the  controlling  taisk  spavmed  N  subtasks  to  perform  the  communication.  This 
has  the  undesirable  side  effect  of  increasing  the  number  of  TCS  as  control 
passes  between  the  main  and  the  subtasks.  If  separate  subtasks  were  used  for 
each  phase,  and  task /subtask  synchronisation  can  be  achieved  by  use  of 
shared  variables,  then  this  requires  at  least  another  TCS,  making  a  total 
of  16N  TCS,  four  times  the  number  required  using  messaige  passing.  It  will  be 
quite  a%<kward  determining  when  each  phase  has  fini^ed  -  perhaps  the  easiest 
way  being  to  use  the  subtask  statusses  to  determine  v*>en  they  have  all 
terminated . 

It  might  be  possible  to  improve  on  these  overhearls  by  creating  N 
subtasks  for  the  duration  of  the  2PC  operation,  however  this  would  mean  that 
we  would  have  to  use  rendezvous  between  the  subtasks  and  the  control  task  in 
order  to  signal  the  end  of  each  phase.  This  still  requires  dN  TCS,  but  wjuld 
involve  less  process  creation  and  destruction  t^ich  must  thenselves  be 
extensive  operations.  However  it  is  possible  that  some  of  these  contexts 
switches  can  be  eliminated  by  performing  optimisations,  analogous  to  the 
Hassi  -  Habermann  optimisations. 

Some  MCS  communication  systems  provide  broadcast  facilities  which  can 
provide  very  efficient  l:many  communication.  The  Ma  rende^ous  does  not 
allow  such  hardware  to  be  exploited. 

In  short  it  seems  that  Ma  will  enforce  very  expensive,  and  quite 
complex  implementations  of  protocols  of  the  above  form.  Since  this  form  of 
protocol  will  be  at  the  heart  of  any  Distributed  Database  Hanager,  and  of 
many  other  applications,  Ma  may  be  a  very  poor  .^ice  of  implementation 
language  firoro  the  point  of  view  of  efficiency,  and  simplicity  of 
implementation . 

0)  Program  Loading  and  Hardware  Mapping 

The  mapping  of  the  Ma  program  onto  the  available  hardware  must  be 
specified  at  some  stage  in  the  program  development  and  loading  process, 
unless  the  maoDing  is  to  be  chosen  automatically  by  the  progranming  support 
environment.  It  ^uld  he  fairly  straightforward  to  specify  the  mapi^ng 


either  as  pcaqmata  in  the  program  source  text,  or  as  oomnanrJs  to  the  program 
loading  system,  although  it  may  be  difficult  to  decide  v*jat  this  mapping 
^uld  be.  We  are  not  concerned  with  the  problems  of  deciding  on  a  mapping, 
rather  on  v*»at  s4>ould  be  loaded,  and  how  it  can  be  executed. 

Tn  particular  we  are  ooncerne-d  with  what  should  happen  to  the  "main 
body"  of  the  program.  Por  the  sake  of  simplicity  let  us  assume  that  the  main 
body  of  the  program  consists  of  a  sequence  of  declarations  of  tasks  v^ich 
are  to  be  run  on  a  number  of  separate  machines.  Clearly  the  code  for  the 
individual  tasks  should  be  loaded  cxi  the  designated  machines  together  with 
code  for  instantiating  the  tasks.  Address  information  to  allow  inter-task 
communication  must  also  be  made  available  to  the  kernel  (this  may  be 
regarded  as  the  vestige  of  the  declarations  of  the  other  tasks) .  We  can  rely 
on  the  semantics  of  the  rendezvous  to  ensure  correct  behaviour  despite  the 
fact  that  tasks  vhich  wish  to  communicate  may  be  created  at  significantly 
different  times. 

The  above  soluticxi  is  satisfactory  so  long  as  no  task  tries  to  create  a 
task  to  tun  in  another  machine.  If  tasks  can  be  created  in  other  machines, 
then  a  mechanism  has  to  be  provided  for  one  kernel  to  request  another  to 
create  and  run  a  task,  this  facility  may  cause  problems  if  the  processes  in 
separate  machines  wish  to  ^are  data,  and  it  may  also  lead  to  difficulties 
in  scheduling,  and  in  assessing  the  amount  of  mill  time  absorbed  by  any 
task,  etc.  Similar  comments  apply  to  the  inclusion  of  executable  code  in  the 
main  body  of  the  program. 

Tt  would  seem  to  be  by  far  the  simplest  if  Ma  programs  to  run  in  MCS 
%iete  restricted  in  the  following  ways,  first,  the  main  body  may  only  consist 
of  the  declaration  of  tasks.  Second,  no  task  may  create  a  task  to  run  on 
another  machine.  Tt  seems  likely  that  these  restrictions  would  be  acceptable 
in  practice. 

10)  Conclusions 

We  have  described  some  possible  ways  in  «bich  an  Ada  rendezvous  could  be 
implemented  in  an  MCS.  WC  h£n;’e  also  considered  the  overheads  of  using  the 
rendezvous  «bere  we  wish  to  perform  1:N,  rather  than  1:1,  communication.  We 
have  ^wn  that  the  overheads  of  using  the  Ma  rende^ous  as  oppsed  to  a 
message  based  communication  system  are  quite  large.  Ibis  is,  perhaps,  not 
surprising  as  procedure  calls  are  fundamental  within  a  single  computer,  but 
message  passing,  rather  than  remote  procedure  calls,  eure  fundamental  to 
communication  systems. 

We  have  briefly  considered  the  problem  of  loading  and  executing  Ma 
programs,  and  we  have  suggested  a  rule  for  constructing  and  mapping  Ma 
programs  «bich  would  simplil^  their  implementation  on  an  MCS. 

We  have  not  covered  all  tiie  important  issues  to  do  with  implementing  Ma 
on  an  MCS.  Por  example  we  have  ignored  problems  of  deciding  on  a  good 
software  to  hardware  mapping;  how  we  test  and  monitor  program  ocecution;  how 
we  extend  or  replace  parts  of  the  running  program  etc.  This  emission  can  be 
iustified  by  saying  that  the  other  problems  are  pertinent  to  other 
programming  languages,  not  iust  Ma.  Nhat  we  have  tried  to  do  is  to 
concentrate  on  tiwae  problems  which  seem  to  be  peculiar  to  Ma. 


